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This  Technical  Brief  describes  objectives,  organization, 
and  methods  that  could  be  used  for  conducting  peer 
reviews  in  public  archeology  projects.  Each  of  these 
elements  is  illustrated  with  examples  drawn  from  peer 
reviews  conducted  by  the  Department  of  the  Interior's 
Departmental  Consulting  Archeologist  between  1981 
and  1990.  The  purpose  of  this  Technical  Brief  is  to 
provide  guidance  to  government  agencies  and  other 
archeological  resources  management  programs  on  use 
of  the  peer  review  process  as  one  tool  to  improve  the 
effectiveness  of  their  activities.  It  may  be  an  especially 
important  tool  to  help  resolve  conflicts  which  may  arise 
due  to  the  need  for  interagency  cooperation,  the  com- 
plexity of  the  archeological  work,  or  professional  dis- 
agreements. This  Technical  Brief  demonstrates  how 
peer  reviews  can  address  these  typical  problems  and 
explains  ways  the  process  can  be  implemented. 

Introduction 

The  review  of  research  by  peers  has  been  one  hall- 
mark of  science  throughout  its  history.  Peer  re- 
views are  the  conscience  of  science;  they  provide  a 
necessary  check  on  its  practitioners  and  an  explicit 
evaluation  of  their  efforts.  Other  professions  have 
recognized  the  importance  of  peer  reviews  for  ap- 
plied sciences  as  well.  For  example,  the  American 
Society  of  Civil  Engineers  and  American  Consult- 
ing Engineers  Council  (ACEC)  have  developed  a 
program  that  offers  peer  review  services  for  engi- 
neering projects  and  programs  (ACEC  1990 
a,b,c,d). 

The  Departmental  Consulting  Archeologist  (DCA) 
and  his  staff  in  the  Archeological  Assistance  Pro- 
gram are  authorized  by  the  Secretary  of  the  Interior 
to  provide  leadership  and  coordination  for  Federal 


and  other  types  of  public  archeology.  The  Secretary 
is  assigned  these  roles  by  several  statutes  including 
the  Antiquities  Act,  Historic  Sites  Act  of  1935, 
National  Historic  Preservation  Act,  and  Archae- 
ological Resources  Protection  Act.  The  peer  review 
of  selected  Federal  archeological  projects  is  one 
among  a  number  of  DCA  programs  that  provide 
leadership  and  coordination  (Canouts  1992; 
CanoutsandMcManamon  1991;  Knoll  1991;  Knud- 
son  and  McManamon  1992;  McManamon,  Smith, 
and  Waldbauer  199C). 

This  Technical  Brief  describes  the  purposes  and  the 
goals  of  the  Federal  archeology  peer  review  process 
as  conducted  by  the  DCA  between  1981  and  1990. 
The  government  agencies  and  the  professionals 
who  have  been  in  these  reviews  recognize  that  peer 
reviews  can  provide  an  important  contribution  to 
project  design  and  management,  thereby  advancing 
American  archeology.  This  brief  provides  an  exam- 
ple of  the  peer  review  process  in  the  Federal  archeo- 
logical arena,  as  opposed  to  the  academic  domain. 
It  also  supplies  information  about  the  planning, 
design,  and  implementation  of  peer  reviews.  Four 
appendices  with  examples  of  documents  illustrating 
efforts  at  planning  and  implementing  peer  reviews 
are  provided. 

Purposes  and  Goals  of  the  Federal 
Peer  Review  Process 

In  an  academic  research  framework,  as  opposed  to 
typical  public  archeology  projects,  the  principal 
investigator  develops  a  research  program  to  inves- 
tigate problems  that  are  of  theoretical,  methodo- 
logical, and/or  substantive  interest.  The  researcher 
develops  a  research  design,  identifies  and  obtains 


permission  to  investigate  the  sites  that  fit  the  pro- 
ject's scientific  needs,  and  raises  the  necessary  funds 
by  obtaining  a  grant.  The  researcher  plans  and 
conducts  the  work  and  publishes  the  results.  Re- 
view of  the  scientific  merits  of  the  proposed  work 
by  his  or  her  peers  occurs  during  the  grant  applica- 
tion process,  in  the  review  of  the  results  by  the 
granting  organization,  as  part  of  the  publication 
process,  and  in  the  reception  of  the  product  by  the 
scientific  community. 

On  the  other  hand,  archeology  conducted  to  com- 
ply with  Federal  historic  and  archeological  preser- 
vation laws  must  follow  sound  archeological 
practice  as  well  as  the  regulations,  procedures,  and 
guidelines  mandated  by  and  developed  from  those 
laws.  Archeology  conducted  by  these  authorities  is 
to  benefit  the  American  people  generally  through 
the  preservation  of  important  archeological  re- 
sources and  information.  The  first  objective  of  re- 
search in  public  archeology  is  to  identify,  evaluate, 
and  record  significant  archeological  resources.  The 
scientific  information  is  used  for  a  variety  of  prac- 
tical purposes,  including  education  and  creating  an 
appreciation  of  the  nation's  heritage  among  its  citi- 
zens. The  concomitant  benefits  that  individual 
archeologists  may  derive  from  these  endeavors  with 
regard  to  professional  standing  and  career  advance- 
ment are  secondary. 

Early  in  the  historic  preservation  process,  the  Fed- 
eral agency  identifies  archeological  properties 
within  the  boundaries  of  an  undertaking.  This  is 
accomplished  through  literature  reviews,  records 
checks,  and  field  surveys.  The  significance  of  the 
sites,  especially  for  the  important  archeological  and 
historical  data  they  contain,  is  determined  based 
upon  eligibility  for  the  National  Register  of  His- 
toric Places.  Significance  commonly  is  assessed  in 
reference  to  State  comprehensive  historic  preserva- 
tion plans  or  historic  contexts  used  by  the  State 
Historic  Preservation  Officer  (SHPO). 

If  important  data  will  be  lost  because  of  the  under- 
taking, the  Federal  agency  must  consider  this  po- 
tential and  consult  with  the  SHPO  to  develop  ways 
to  mitigate  the  loss.  The  agency  also  may  have  to 
get  the  comments  of  the  Advisory  Council  on 
Historic  Preservation  (ACHP)  before  moving  for- 
ward with  the  undertaking. 


Several  parameters  have  been  defined  for  Federal 
archeological  projects  that  constrain  the  scientific 
freedom  to  develop  research  topics.  The  principal 
difference  between  an  exclusively  academic  re- 
search design  and  one  developed  for  a  public  project 
is  that  the  former  develops  from  scientific  curiosity 
while  in  the  latter  scientific  research  needs  are  con- 
ditioned by  the  potential  data  content  of  the  archeo- 
logical  site  being  impacted  by  the  public 
undertaking. 

These  constraints  on  the  opportunity  for  an  indi- 
vidual researcher  to  develop  theoretical,  methodo- 
logical, or  substantive  aspects  of  the  research  design 
do  not  mean  that  Federal  projects  are  devoid  of  such 
concerns.  On  the  contrary,  the  process  of  develop- 
ing a  research  design  and  a  strategy  to  implement  it, 
completing  the  work,  and  reporting  are  the  same 
general  steps  of  any  scientific  research  project.  Also, 
the  development  of  the  archeological  project  within 
Federal  historic  preservation  is  a  consultative  proc- 
ess in  the  hands  of  Federal  archeologists  and  their 
counterparts  in  the  SHPO  office.  Project  plans  and 
archeological  results  also  may  be  reviewed  by 
archeologists  and  historic  preservation  profession- 
als of  the  ACHP.  Frequently  suggestions  made 
during  these  consultations  result  in  design  improve- 
ments. As  a  result  of  this  coordination,  the  project 
has  a  firm  scientific  foundation. 

Before  the  passage  of  the  National  Historic  Preser- 
vation Act  of  1966  (NHPA),  rescue  or  salvage  ar- 
cheology was  handled  without  input  from  the 
States  or  the  ACHP.  Archeological  rescue  work 
was  usually  controlled  by  the  time  and  money 
available  rather  than  by  the  development  of 
thoughtful  and  explicit  research  designs.  The  Na- 
tional Park  Service  (NPS),  operating  with  its  own 
professional  staff  through  the  River  Basin  Survey 
program  of  the  Smithsonian  Institution  or  with 
academic  contractors,  conducted  the  archeological 
salvage  program  of  the  nation.  Private  energy  de- 
velopers and  State  highway  departments  contracted 
directly  with  academic  institutions  to  complete  sal- 
vage archeology  in  their  construction  projects. 
Competition  among  academic  institutions  for  gov- 
ernment, private  utility  company,  or  highway  de- 
partment contracts  for  archeological  work  did  not 
exist  during  this  period.  Normally  the  NPS,  the 
utility  company,  or  highway  department  ap- 
proached an  academic  institution  and  asked  the 
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institution  to  carry  out  whatever  archeology  was 
needed  within  the  time  and  funding  available. 

Today  Federal  agency  archeologists  or  contractors 
from  the  public  and  private  sectors  conduct  Federal 
archeological  projects.  Competition  among  archeo- 
logical  contractors  is  the  norm  unless  the  work  is 
done  by  agency  personnel.  It  does  not  matter  who 
does  the  project  for  the  Federal  peer  review  process 
to  work. 

The  perception  has  existed  that  Federal  archeologi- 
cal project  research  design  is  less  open  to  evaluation 
and  criticism  than  an  academic  research  project 
funded  by  a  granting  agency.  This  is  hardly  the  case. 
As  mentioned,  evaluations  occur  as  part  of  memo- 
randa of  agreement,  National  Register  of  Historic 
Places  nominations,  determinations  of  effect,  re- 
views of  scopes  of  work  for  data  recovery  plans,  and 
requests  for  proposals  that  form  the  bases  for  con- 
tracts. Depending  on  the  specifics  of  the  project, 
these  evaluations  are  performed  by  archeologists, 


other  cultural  resources  specialists,  and  perhaps 
officials  of  various  agencies.  These  reviews  repre- 
sent a  broad  range  of  scientific  and  legal  compliance 
concerns.  Frequently  the  evaluations  improve  the 
agency  proposal  to  mitigate  the  loss  of  important 
information,  and  stipulations  for  formal  peer  re- 
views are  often  specifically  included  in  memoranda 
of  agreement  or  agency  contracts.  This  is  all  done 
before  a  project  is  announced  for  bidding.  As  part 
of  the  proposal,  the  winning  bidder  also  prepares  a 
research  design  or  work  plan.  This  provides  refine- 
ments for  conducting  the  research.  It  also  defines 
the  protocols  that  will  serve  as  project  guides. 

The  primary  goal  of  the  peer  review  process  devel- 
oped by  the  DCA  is  to  evaluate  the  conduct  of 
Federal  archeological  projects  and  assess  the  com- 
petence and  efficiency  of  projects  relative  to  archeo- 
logical practice  and  legal  compliance.  An  academic 
peer  review  rates  a  project  on  its  scientific  merits 
alone;  reviewers  examine  the  theoretical  basis  for 
the  research,  the  data  needed,  and  how  they  con- 


tribute  to  the  research  goals.  They  evaluate  the 
methodology  used  to  get  needed  data  and  the  ana- 
lytical methodology.  They  determine  if  the  re- 
search was  conducted  in  an  appropriate  manner.  In 
Federal  archeology,  the  peer  review  examines  addi- 
tional topics  as  well  as  these.  Is  the  agency  following 
the  stipulations  of  the  compliance  documents 
agreed  to  under  the  Section  106  process  of  NHPA? 
Have  data  fulfilled  the  needs  of  the  research  design? 
Have  unexpected  data  or  research  domains  come  to 
light  that  need  additional  investigation?  Does  the 
project  need  to  be  changed  or  redesigned  to  take 
these  findings  into  consideration? 

During  the  1980s,  Federal  archeology  project  peer 
reviews  were  undertaken  when  controversy  stirred 
about  an  undertaking  or  the  archeology  associated 
with  it.  Some  reviews  were  conducted  because  of 
major  disagreement  between  the  agency  and  the 
contractor  about  the  work.  Allegations  or  ques- 
tions may  have  come  from  the  archeological  profes- 
sion, the  media,  special  interest  groups,  or  members 
of  Congress.  More  recently  agencies  have  requested 
reviews  by  the  DC  A  to  investigate  whether  projects 
were  adequate.  One  review  was  conducted  for  an 
agency  that  wished  to  improve  a  project. 

In  public  archeology  the  use  of  the  peer  review 
process  also  serves  other  purposes: 

(1)  Demonstrating  to  the  professional  community, 
Congress,  the  Administration,  and  others  that  the 
Federal  archeology  program  produces  excellent 
results; 

(2)  Creating  networks  for  communication  between 
government  and  academic  archeologists  that 
improve  archeology; 

(3)  Helping  resolve  disputes  between  Federal 
agencies,  contractors,  special  interest  groups,  and 
the  media; 

(4)  Providing  Federal  archeologists  with 
professional  credibility  among  their  managers  and 
the  academic  community. 

Philosophy  of  Peer  Reviews  for 
Public  Projects 

In  designing  the  peer  review  process  the  scientific 
value  of  an  archeological  project,  as  determined  by 
the  NHPA  Section  106  consultation,  is  accepted 


and  used.  Furthermore,  the  research  goals  and  pri- 
orities and  the  stated  applications  of  archeological 
methodology  also  are  accepted  and  used.  The  peer 
review  evaluates  the  project  using  the  following 
criteria: 

(1)  Is  the  archeological  research  consistent  with  the 
needs  of  the  research  design? 

(2)  Is  it  consistent  with  the  contract,  memorandum 
of  agreement,  and  other  covering  directives? 

(3)  Is  the  project  on  schedule? 

(4)  Is  the  project  within  budget? 

(5)  Are  the  agency  and  the  contractor  fulfilling  the 
requirements  of  the  contract? 

(6)  Are  there  unresolved  disputes  or  disagreements 
between  the  contractor  and  the  agency  related  to 
the  archeological  work? 

(7)  Are  the  materials  and  data  recovered  consistent 
with  the  needs  of  the  research  design? 

(8)  Are  there  gaps  in  the  data?  If  so,  is  it  possible  to 
fill  them? 

(9)  Is  it  desirable  to  suspend  a  particular  line  of 
research? 

(10)  Based  upon  new  findings,  should  other  lines  of 
research  be  pursued  in  place  of  or  in  addition  to 
those  identified  in  the  research  design? 

(11)  Are  the  public  education  and  outreach  aspects 
of  the  project  being  done  adequately  and 
effectively? 

(12)  Do  the  steps  being  taken  to  clean,  catalog, 
describe,  and  analyze  excavated  material  conform 
to  the  plans  for  ultimate,  long-term  curation  of 
the  material  and  data? 

This  approach  accepts  previous  decisions  made  by 
the  various  consulting  parties  to  mitigate  the  loss  of 
important  archeological  information  caused  by  a 
Federal  undertaking.  It  also  provides  an  occasion  to 
assess  the  project  to  determine  if  any  changes  are 
desirable.  This  approach  steers  clear  of  second- 
guessing  earlier  decisions  or  trying  to  identify  all  of 
the  approaches  for  conducting  archeological  inves- 
tigations. The  manager  of  the  peer  review  and  the 
agency  archeologist  establish  a  project  review 
agenda  and  develop  a  list  of  suggested  review  topics. 
These  aid  the  peer  review  panel  in  examining  arche- 
ology defined  by  the  NHPA  Section  106  process. 


Table  1.   Projects  for  Which  Peer  Reviews  Were  Done  by  the  Departmental  Consulting  Archeologist,  Archeological 
Assistance  Division,  National  Park  Service 


Project/Loca  tion 


Year 


Agency 


Dolores  Project,  McPhee  Reservoir,  CO 

Central  Arizona  Project,  AZ 

Jackson  Lake  Dam,  Grand  Tetons  National  Park,  WY 

Stillwater  Wildlife  Management  Area,  Carson  Desert,  NV 

Libby  Dam  Project,  Lake  Koocanusa,  MT 

Alkali  Creek  Project,  ND 


1981 
1986 
1987 
1988 
1989 
1990 


Bureau  of  Reclamation 
Bureau  of  Reclamation 
National  Park  Service 
Fish  and  Wildlife  Service 
Corps  of  Engineers 
Soil  Conservation  Service 


Project  Selection 

Since  198 1,  the  DCA  has  conducted  six  peer  reviews 
(Table  1).  The  experience  from  conducting  these 
reviews  is  the  basis  for  this  Technical  Brief.  These 
projects  were  selected  for  a  variety  of  reasons.  The 
Dolores  Reservoir  Project  was  reviewed  because  of 
problems  between  the  contractor  and  the  agency 
about  the  scope  of  the  investigation  and  project 
management.  The  Central  Arizona  Project  was  se- 
lected because  the  Bureau  of  Reclamation  wanted 
to  confirm  that  the  scope  and  focus  of  the  archeo- 
logical project  were  appropriate.  The  Jackson  Lake 
Dam  Project  was  evaluated  to  demonstrate  to  the 
members  of  the  Wyoming  Congressional  delega- 
tion that  the  work  was  adequate  and  satisfied  State 
interests.  The  Fish  and  Wildlife  Service  sought  a 
peer  review  to  address  Native  American  concerns 
and  to  assure  that  historic  preservation  require- 
ments were  being  met  in  the  Stillwater  Wildlife 
Management  Area  Project.  The  U.S.  Army  Corps 
of  Engineers  wanted  to  confirm  that  its  historic 
preservation  responsibilities  were  being  fulfilled  in 
a  manner  sensitive  to  Native  American  religious 
concerns  in  the  Libby  Dam  Project.  The  Soil  Con- 
servation Service  wanted  an  evaluation  of  the  Alkali 
Creek  Project  in  light  of  SHPO  and  professional 
community  concerns. 

Organizing  and  Conducting 
Peer  Reviews 

A  peer  review  of  a  project  is  an  activity  that  must 
be  developed  harmoniously  as  a  partnership  be- 
tween the  peer  review  manager  and  the  agency.  For 


all  but  the  Alkali  Creek  peer  review,  the  DCA 
served  as  peer  review  manager.  In  the  case  of  Alkali 
Creek,  a  senior  NPS  archeologist  well  acquainted 
with  the  project  area  was  designated  by  the  DCA 
as  peer  review  manager.  Normally  the  agency  rep- 
resentative has  been  the  archeologist  responsible  for 
the  administration  and  conduct  of  the  agency's 
archeological  program,  although  another  agency 
official  may  serve  in  this  role.  The  need  for  frequent 
and  clear  communication  between  the  peer  review 
manager  and  the  agency  archeologist  or  other  offi- 
cial in  charge  of  the  project  is  mandatory.  Good 
communication  is  an  absolute  necessity  for  a  suc- 
cessful peer  review.  No  matter  who  serves  in  them, 
these  roles  are  essential  components  of  the  peer 
review  process. 

All  aspects  of  the  peer  review  are  developed 
through  close  coordination  between  the  peer  re- 
view manager  and  the  agency.  Both  parties  must 
agree  on  the  scope  of  the  review.  Frequent  and  open 
communication  is  necessary  in  all  aspects  of  the 
project.  The  agency  archeologist  needs  information 
about  the  progress  of  the  planning  effort  by  the  peer 
review  manager  to  develop  agency  plans.  The  peer 
review  manager  also  discusses  the  peer  review  with 
the  SHPO  and  ACHP  to  gather  information  that 
should  be  supplied  to  the  team.  Frequent  commu- 
nication bolsters  the  commitment  made  by  both 
parties  to  assure  that  the  necessary  actions  are  com- 
pleted in  a  timely  manner.  In  undertaking  a  peer 
review,  one  of  the  peer  review  manager's  goals  is  to 
create  a  partnership  with  the  agency.  The  creation 
of  such  a  partnership  is  the  principal  ingredient  that 


has  made  the  process,  as  conducted  by  the  DCA, 
successful. 

Once  a  project  is  selected,  the  following  tasks,  at  a 
minimum,  must  be  undertaken  to  conduct  the  peer 
review: 

(1)  Defining  the  review's  scope  and  schedule. 

(2)  Determining  the  review's  cost  and  securing 
funds. 

(3)  Planning  the  team's  composition  and  choosing 
members. 

(4)  Compiling  documentation  and  mailing  to  the 
team. 

(5)  Scheduling  the  on-site  visit. 

(a)  Arranging  travel  and  lodging. 

(b)  Reserving  meeting  space. 

(c)  Making  appointments  for  interviews. 

(d)  Preparing  the  agenda  and  itinerary. 

(e)  Identifying  review  topics. 

(f)  Developing  support  for  the  team. 

(g)  Making  arrangements  for  office  space, 
(h)  Securing  equipment  for  team  use. 

(6)  Conducting  the  on-site  visit. 

(7)  Preparing  the  draft  final  report  (by  peer  review 
manager). 

(a)  Obtaining  revisions  to  the  final  draft 
report  from  the  review  team. 

(b)  Revising  manuscript. 

(c)  Obtaining  review  team  approval  of  final 
report. 

(d)  Producing  the  final  report. 

(e)  Compiling  recommendations  for  the 
agency. 

(f)  Completing  remaining  administrative 
tasks. 

(8)  Providing  final  report  to  agency  officials. 

The  following  sections  of  this  Technical  Brief  will 
provide  information  about  each  of  these  tasks.  The 
examples  cited  are  drawn  from  peer  reviews  con- 
ducted in  the  past  by  the  DCA. 

1.  Defining  the  Scope  of  the  Peer  Review 

The  reasons  for  selecting  a  project  for  peer  review 
normally  defines  the  scope  of  the  review.  If  charges 
of  inadequacy  have  been  made,  the  peer  review  will 
focus  on  the  conduct  of  the  archeological  investiga- 
tions under  the  agreements  and  documents  cover- 


ing the  project.  If  the  peer  review  is  a  response  to 
problems  between  the  agency  and  the  contractor, 
the  peer  review  will  focus  on  these  matters.  The 
topics  identified  for  review  for  the  Stillwater  Wild- 
life Management  Area  are  provided  as  examples  in 
Appendix  A;  those  for  the  Jackson  Lake  Dam  pro- 
ject were  similar.  In  any  case,  the  scope  of  the  peer 
review  should  focus  on  identifying  solutions  to 
problems  and  not  foster  inappropriate  or  adversar- 
ial investigative  roles. 

2.  Cost  of  the  Peer  Review  and  Funding 

Arrangements  must  be  made  to  fund  the  various 
costs  for  peer  reviews  and  establish  the  budget 
parameters  within  which  the  peer  review  manager 
must  operate.  The  costs  of  peer  reviews  conducted 
by  the  DCA,  for  instance,  have  been  covered  by 
various  means,  including  the  DCA's  annual  operat- 
ing budget,  shared  expenses  of  the  DCA  and  the 
agency,  and  full  funding  by  the  agency.  When  an 
agency  shares  or  covers  the  costs,  the  funds  typi- 
cally come  from  the  project  budget.  Cost  sharing 
between  the  DCA  and  the  agency  has  been  the  most 
common  arrangement. 

Honoraria  or  consulting  fees  have  not  been  paid  to 
members  of  the  DCA  peer  review  panels.  Payment 
of  honoraria  to  non-Federal  members  of  the  team 
is  legal,  but  it  is  illegal  to  pay  a  Federal  team 
member.  It  seemed  inappropriate  to  compensate 
some  team  members  but  not  others.  Furthermore, 
there  was  concern  that  if  honoraria  were  paid  the 
peer  review  team  might  not  feel  that  it  could  be 
critical  and  candid  in  its  assessment.  There  was  also 
a  potential  for  the  review  to  appear  prejudiced. 

The  costs  of  transportation  and  per  diem  of  the  peer 
review  members  must  be  considered.  The  purchase 
of  airline  tickets  may  take  a  large  portion  of  the 
funds  available.  Other  costs  include:  reproduction 
and  postage  to  provide  project  information  to  the 
peer  review  team;  on-site  equipment,  materials,  and 
work  space  for  the  team;  administrative  support; 
and  final  report  preparation  and  distribution.  Many 
of  these  costs  can  be  covered  in-kind  by  the  agency 
or  the  peer  review  manager. 

3.  Composition  and  Selection  of  the 
Peer  Review  Team 

The  first  peer  review  team  assembled  by  the  DCA 
consisted  of  three  nationally  recognized  archeolo- 


gists  from  academic  positions.  Two  were  experts  in 
the  archeology  of  the  project  region.  One  was  an 
expert  from  outside  the  area.  This  composition  was 
modified  for  later  peer  reviews.  It  was  learned  that 
academic  archeologists  were  unfamiliar  with  the 
NHPA  Section  106  process  and  unaware  of  many 
of  the  legal  requirements  of  Federal  historic  preser- 
vation. They  had  little  or  no  knowledge  of  Federal 
procurement  and  contracting  regulations  and, 
therefore,  they  found  it  difficult  to  readily  identify 
efficient,  effective  solutions  to  problems  related  to 
these  topics. 

For  later  reviews,  an  archeologist  from  a  public 
agency,  but  one  different  from  the  one  under  re- 
view, was  added  to  the  team.  This  served  the  addi- 
tional goal  of  increasing  the  interaction  among 
government  archeologists  and  their  academic  col- 
leagues. This  was  a  way  to  improve  the  credibility 
of  the  government  archeologists  with  their  aca- 
demic counterparts.  We  have  found  that  a  balanced 
peer  review  teams  consists  of  one  public  archeolo- 
gist from  an  agency  other  than  the  one  responsible 
for  the  project,  an  academic  expert  in  the  archeol- 
ogy of  the  project  area  or  region,  and  an  academic 
expert  from  outside  the  research  area. 

To  identify  potential  review  team  members  the 
peer  review  manager  and  the  agency  archeologist 
independently  compile  three  lists  of  candidates  for: 
(1)  the  regional  academic  expert,  (2)  the  outside 
academic  expert,  and  (3)  the  public  agency  arche- 
ologist. They  consolidate  their  lists  and  decide 
which  candidates  and  alternates  to  invite. 

Care  is  to  be  given  in  selecting  members  of  the 
panel.  Beyond  a  candidate's  scientific  credentials 
and  availability,  other  elements  are  considered,  for 
example: 

(1)  Does  the  candidate  have  a  reputation  for 
completing  work  in  a  reasonable  time? 

(2)  Does  the  candidate  have  a  previous  relationship 
or  any  other  consideration  with  the  agency  or  the 
project  that  could  be  construed  as  a  conflict  of 
interest? 

(3)  Was  the  candidate  an  unsuccessful  bidder  for  the 
contract? 

(4)  Does  the  candidate  work  well  with  others? 


While  these  elements  may  seem  straightforward, 
ignoring  them  could  make  completing  an  accept- 
able peer  review  difficult  or  affect  its  timeliness. 

Once  the  peer  review  manager  and  agency  arche- 
ologist agree  on  the  candidates  and  the  approximate 
dates  for  the  review,  the  peer  review  manager  tele- 
phones the  candidates  to  determine  their  willing- 
ness and  availability.  Acceptance  of  the  invitation 
is  followed  by  a  formal  letter  from  the  peer  review 
manager  providing  information  about  the  project, 
the  schedule,  and  other  details. 

4.  Compiling  Documentation  for  the 
Peer  Review  Team 

By  the  time  of  the  on-site  visit  the  peer  review  team 
should  have  a  thorough  knowledge  of  the  project. 
The  agency  responsible  for  the  project  compiles  and 
furnishes  the  panel  with  the  documents  that  de- 
scribe the  undertaking,  the  archeological  resources, 
their  significance,  the  research  questions,  and  back- 
ground information  about  the  local  archeology. 
Acquisition  of  this  necessary  knowledge  by  the  peer 
review  team  is  facilitated  by  providing  all  the  docu- 
mentation to  team  members  at  least  three  weeks 
before  the  on-site  visit.  If  it  is  provided  with  less 
time  before  the  on-site  visit  the  peer  review  team 
will  not  have  time  to  review  this  basic  information. 

The  package  of  material  compiled  for  the  orienta- 
tion of  the  peer  review  team  will  vary  from  project 
to  project.  Typically  it  will  consist  of  survey  and 
evaluation  reports  and  NHPA  Section  106  docu- 
mentation, such  as  the  Memorandum  of  Agreement 
or  Determination  of  No  Adverse  Impact,  scope  of 
work  or  request  for  proposal,  and  research  design. 
Interim  or  annual  progress  project  reports,  if  avail- 
able, also  are  provided.  Additionally,  correspon- 
dence and  internal  documents  may  be  included.  (A 
list  of  the  kinds  of  materials  supplied  for  the  Central 
Arizona  Project  and  Stillwater  National  Wildlife 
Management  Area  peer  reviews  is  provided  in  Ap- 
pendix B.) 

5.  Scheduling  the  Peer  Review 

The  ideal  time  for  a  peer  review  is  at  the  midpoint 
of  a  project.  Typically,  field  and  laboratory  work 
has  proceeded  to  the  stage  where  the  review  team 
can  make  a  reasonable  evaluation  of  the  results.  A 
peer  review  at  the  beginning  of  a  project  is  prema- 
ture because  there  will  be  few  results  to  assess.  If  the 
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project  is  near  completion  the  important  recom- 
mendations of  the  peer  review  team  will  be  difficult 
to  implement;  usually  time  and  money  have  be- 
come short  at  this  stage. 

A  second  consideration  in  scheduling  the  peer  re- 
view is  for  a  time  that  is  acceptable  to  potential  team 
members.  It  is  useful  to  identify  a  3-  to  4-week 
period  for  the  peer  review  approximately  3  months 
in  advance  of  the  on-site  visit.  Within  such  a  time 
frame  it  is  usually  possible  to  fit  the  schedules  of  all 
personnel  involved.  Situations  that  can  affect  the 
scheduling  of  the  review  must  be  considered.  These 
include  college  and  university  calendars  or  special 
events  like  conventions  or  athletic  contests  in  the 
project  area.  These  situations  can  affect  travel  ar- 
rangements, as  lodging  may  be  difficult  to  find  in 
such  circumstances.  The  schedules  and  commit- 
ments of  the  individuals  representing  the  various 
organizations  who  will  be  interviewed  by  the  peer 


review  team  must  be  considered.  The  on-site  visit 
must  be  scheduled  so  that  those  who  will  travel 
from  outside  the  project  area,  the  SHPO  or  appro- 
priate staff,  ACHP  personnel,  and  other  interested 
parties,  can  participate. 

The  peer  review  manager,  in  consultation  with  the 
agency  archeologist,  develops  an  agenda  for  the  peer 
review  team's  on-site  visit.  A  detailed  agenda  assures 
that  the  on-site  visit  is  carefully  structured.  Each 
interview  focuses  on  a  specific  task  or  topic.  Each 
interview  session  is  allotted  a  specific  amount  of 
time.  Sessions  usually  do  not  exceed  two  hours 
without  a  break.  Maintaining  strict  adherence  to 
the  agenda  assures  that  the  peer  review  team  stays 
focused  on  the  task  and  adheres  to  the  schedule. 

During  the  first  session  of  the  on-site  visit  the  peer 
review  manager,  the  agency  archeologist,  and  the 
peer  review  team  discuss  the  agenda  in  detail  and 
modify  it  as  necessary.  Implementing  modifications 
in  the  agenda  may  require  complex  changes  in  the 
interview  schedule.  It  is  important  to  determine  the 
final  agenda  as  early  as  possible. 

6.  On-Site  Visit 

The  peer  review  manager  and  the  agency  archeolo- 
gist plan  the  on-site  visit  to  use  everyone's  time 
efficiently.  Usually  the  peer  review  manager, 
agency  personnel,  and  peer  review  team  arrive  the 
day  before  the  on-site  visit  begins. 

The  peer  review  manager  and  the  agency  archeolo- 
gist provide  the  necessary  administrative  support  to 
the  peer  review  team.  This  assures  that  an  efficient 
review  occurs.  Meeting  facilities,  telephones,  trans- 
portation, and  lodging  and  dining  facilities  must  be 
available.  Secretarial  assistance,  equipment,  and 
supplies  may  be  needed.  The  object  of  taking  care 
of  these  necessities  is  to  allow  the  team  to  devote  its 
energies  to  the  peer  review.  Personal  computers 
have  been  very  useful.  In  providing  these  kinds  of 
administrative  support,  the  peer  review  manager 
and  agency  function  as  aides  to  the  team. 

The  first  part  of  the  agenda  is  devoted  to  briefings 
by  the  peer  review  manager  and  agency  personnel. 
The  peer  review  manager  describes  and  then  em- 
phasizes the  specific  scope  of  work  of  the  peer 
review  (see  Appendix  C  for  examples).  The  peer 
review  team  can  examine  any  aspects  of  the  project 


related  to  the  archeological  work.  The  plan  for  the 
rest  of  the  on-site  visit  will  depend  on  the  schedules 
of  individuals,  the  weather,  field  conditions,  and 
other  variables.  During  the  week-long  peer  reviews 
conducted  by  the  DCA,  personnel  interviews  and 
field  visits  normally  were  concluded  by  the  end  of 
the  third  day  so  the  team  could  review  its  findings 
and  draft  its  report.  Typically  the  draft  report  was 
completed  by  the  evening  of  the  fourth  day.  Exit 
interviews  with  the  DCA,  agency  personnel,  and 
other  interested  parties  were  conducted  on  the  fifth 
day. 

The  peer  review  team  normally  interviews  agency 
personnel  such  as  the  project  engineer,  environ- 
mental staff,  and  the  archeologist.  Other  historic 
preservation  specialists,  the  contracting  officer,  or 
contracting  officer's  technical  representative  may 
be  interviewed.  The  team  meets  with  the  contractor 
or  principal  investigator  and  the  principal  investi- 
gator's senior  staff,  such  as  field  directors,  crew 
chiefs,  data  control  manager,  and  laboratory  direc- 
tor. The  team  also  meets  with  the  interdisciplinary 
consultants,  such  as  geologists,  geomorphologists, 
pedologists,  paleobotanists,  paleozoologists,  and 
physical  anthropologists.  The  team  usually  will 
visit  the  site  or  sites  currently  under  investigation 
and  the  laboratory  facilities  of  the  contractor  to  get 
a  feel  for  the  day-to-day  operation  and  management 
of  the  project.  In  some  instances  the  peer  reviewers 
will  meet  with  some  members  of  the  professional 
community  and  Indian  Tribes  or  their  repre- 
sentatives. The  team  also  may  interview  interested 
members  of  the  public  and  Congressional  staffers. 

Each  peer  review  will  vary  regarding  the  parties 
interviewed  because  the  focus  of  the  peer  review 
will  be  different  in  each  project.  In  the  peer  reviews 
conducted  by  the  DCA,  it  was  standard  procedure 
to  provide  the  SHPO  and  ACHP  with  a  chance  to 
meet  with  the  peer  review  team.  Sometimes  this 
was  done  by  telephone  interviews.  This  considera- 
tion is  based  on  the  regulatory  roles  of  these  agen- 
cies in  historic  preservation.  Furthermore,  the 
professional  staff  of  the  SHPO  are  especially  knowl- 
edgeable about  their  State's  archeological  resources 
and  can  provide  important  perspectives  to  the  team. 

The  scope  of  the  peer  review  and  the  specific  cir- 
cumstances will  determine  whether  or  not  the 
agency  archeologist  or  official  attends  all  of  the 


interviews  conducted  by  the  team.  In  cases  where 
problems  exist  between  the  agency  and  the  contrac- 
tor, SHPO,  ACHP,  or  other  parties,  it  has  been 
found  prudent  for  the  team  to  meet  both  with  and 
without  the  agency  personnel.  The  absence  of 
agency  personnel  may  allow  for  open  and  candid 
discussion  that  otherwise  would  be  difficult. 

The  on-site  visit  is  a  period  of  intense  work  by  the 
peer  review  team.  Every  effort  is  made  to  keep  the 
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team  on  schedule  and  focused  on  its  task.  The  peer 
review  manager  and  agency  personnel  provide  all 
reasonable  support  requested  to  assure  that  the 
team's  tasks  are  completed  by  the  scheduled  time. 

7.  Peer  Review  Draft  Report 

The  peer  review  manager  and  the  agency  archeolo- 
gist prepare  a  proposed  outline  for  the  report,  mak- 
ing it  clear  to  the  peer  review  team  that  the  topics 
and  proposed  outline  are  only  suggestions.  The 
content  of  the  final  report  is  determined  by  the  peer 


review  team.  The  purpose  of  the  proposed  report 
outline  is  to  provide  the  team  with  guidance.  The 
interviews  provide  specific  and  important  informa- 
tion and  views  from  a  variety  of  perspectives.  The 
peer  review  manager  always  assures  the  team  in 
each  review  that  the  content  and  recommendations 
of  the  final  report  are  entirely  their  responsibility. 
The  team  is  welcome  to  change  or  reorganize  the 
outline  and  format.  Team  members  may  add  or 
drop  topics  as  they  think  necessary  or  appropriate, 
though  departures  from  the  model  usually  concern 
format  or  organization. 

The  importance  of  coordination  and  careful  atten- 
tion to  details  in  production  of  the  final  draft  report 
cannot  be  overestimated  by  the  peer  review  man- 
ager. Some  cautionary  examples  are  cited  from  the 
peer  reviews  conducted  by  the  DCA  to  demon- 
strate the  importance  of  organized,  considered  ac- 
tivities and  then  continuing  to  learn  from 
experience. 

The  peer  review  team  for  the  Dolores  Project  at 
McPhee  Reservoir  spent  an  intensive  week  inter- 
viewing various  parties  concerned  with  the  project. 
After  the  on-site  visit  the  team  members  returned 
to  their  regular  employment  and  prepared  the  draft 
final  report.  As  a  long-distance  effort,  completion 
of  the  draft  was  delayed  considerably.  This  delay 
was  caused  by  factors  such  as  lack  of  communica- 
tion between  the  members  of  the  team,  the  time 
taken  to  circulate  drafts  and  comments  sequentially 
through  the  mail,  and  other  commitments  or  pri- 
orities of  the  panel  members.  The  Dolores  Project 
peer  review  team  was  not  provided  with  a  proposed 
report  outline,  so  team  members  had  to  develop  an 
outline  and  report  format,  which  proved  to  be  very 
time-consuming.  Based  on  this  experience  the  DCA 
determined  it  would  be  helpful  in  the  future  to 
provide  guidance  and  suggested  topics  to  cover. 
This  practice  has  expedited  the  preparation  of  draft 
peer  review  reports. 

The  Central  Arizona  Project  peer  review  team 
decided  that  it  would  complete  a  draft  report  by  the 
end  of  the  on-site  visit.  Despite  certain  misgivings, 
this  proposal  was  encouraged  and  the  draft  report 
was  delivered  as  promised.  The  DCA  edited  the 
draft  report  for  grammar  and  worked  with  the  for- 


mat, but  made  no  changes  in  content.  A  revised 
draft  was  mailed  simultaneously  to  the  authors  for 
revision.  Marked  and  revised  copy  was  returned  to 
the  DCA  and  changes  made.  The  final  report  was 
produced  and  distributed  to  the  Bureau  of  Reclama- 
tion in  a  short  period  along  with  the  DCA's  own 
recommendations.  Other  interested  parties  were 
sent  copies  as  part  of  this  distribution.  The  Central 
Arizona  Project  peer  review  demonstrated  the  clear 
value  of  requiring  the  team  to  complete  a  draft 
report  by  the  end  of  the  on-site  visit. 

Appendix  D  provides  examples  of  report  outlines 
presented  to  the  peer  review  teams  for  the  Central 
Arizona  and  Carson  Desert  projects. 

8.  The  Final  Report 

The  production  of  the  final  report  of  the  peer 
review  can  be  a  rather  simple,  but  intense  task.  The 
draft  report  is  edited  for  style  and  format  and 
unclear  language  is  revised.  All  changes  in  the 
manuscript  are  made  using  the  "strikeout,"  "bold" 
or  other  options  of  the  chosen  word  processing 
program.  This  allows  the  author  to  identify  easily 
the  changes  that  have  been  made  in  the  revised  text. 
This  draft  is  mailed  simultaneously  to  the  authors 
for  their  corrections  and  approval.  The  peer  review 
manager  makes  the  changes  required  by  the  authors 
and  produces  the  final  report.  Depending  on  the 
quality  of  the  draft  report,  production  of  the  final 
version  requires  some  16  to  24  person  hours  of 
effort. 

The  peer  review  manager  transmits  the  report  and 
recommendations  for  action  to  the  agency  for  con- 
sideration. The  peer  review  manager  may  wish  to 
include  his  or  her  own  recommendations,  which 
address  the  findings  and  recommendations  of  the 
peer  review  team  and  may  be  mandated  according 
to  the  authorities  by  which  the  peer  review  man- 
ager undertook  this  task.  While  the  peer  review 
manager  normally  endorses  most  of  the  recommen- 
dations made  by  the  peer  review  team,  occasionally 
it  will  be  determined  that  the  recommendations  do 
not  really  improve  the  project  when  compared  to 
costs  by  the  agency  to  implement  them.  Irrespec- 
tive of  whether  the  peer  review  manager  supports 
or  rejects  the  recommendations  of  the  team,  a  clear 
explanation  is  always  provided. 
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Conclusion 

Peer  reviews  for  public  archeology  projects  have 
been  and  will  continue  to  be  important  tools  for 
agencies  to  use  to  improve  both  research  and  pres- 
ervation activities.  They  assist  agencies  in  meeting 
the  objectives  for  which  public  archeology  projects 
are  undertaken.  In  addition,  they  can  help  improve 
public  awareness  of  the  value  of  contributions  to 
knowledge  about  the  nation's  cultural  past  that  are 
central  to  such  projects.  When  peer  reviews  are 
conducted  in  this  comprehensive  way  they  have 
had  two  important  long-term  results.  First,  the 
agency  gains  a  better  perspective  on  the  impacts  of 
sound  archeological  resources  management  and 
therefore  can  develop  effective  means  to  improve 
its  programmatic  efforts.  Second,  the  benefits  of 


public  archeology  research  are  clarified  for  the  aca- 
demic disciplines  and  therefore  can  be  incorporated 
appropriately  into  the  most  current  theoretical  and 
interpretive  developments  for  explaining  the  ar- 
cheological record. 

Implicit  also  in  the  purpose  of  this  Technical  Brief 
to  provide  guidance  to  governmental  agencies  and 
other  archeological  resources  management  pro- 
grams on  the  peer  review  process  is  encouragement 
for  those  agencies  to  undertake  their  own.  Such 
peer  reviews  may  be  full-scale  versions,  similar  to 
ones  completed  by  the  DCA,  or  they  may  be 
smaller,  to  fit  the  needs  of  more  localized  projects. 
In  either  case,  the  results  can  lead  to  cost  savings  and 
increased  support  for  archeology  among  members 
of  the  public  and  within  agencies. 
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APPENDIX  A 

Suggested  Peer  Review  Topics  from  Carson  Desert  Archeological  Project  Peer  Review 

(Stillwater  Wildlife 

Management  Area) 

♦  History  of  archeological  project  development 

♦  Status  and  summary  of  past  archeological  work 

♦  Management  and  administration  of  project 

♦  Public  benefits 

♦  Project  planning  and  sequencing 

♦  Volunteer  program 

♦  Interagency/group  cooperation  and  coordination 

♦  ("In  rat  ion 

♦  Compliance  with  Section  106,  National  Historic 

▼           V f  Ul     rM      1   V     '    1     1 

Preservation  Act — how  handled 

♦  Dissemination  of  reports  to  professional 

♦  Maintenance  of  quality  control 

community 

♦  Budget  and  financing  of  archeological  program, 

♦  Value  of  peer  review 

estimate,  cumulative  cost,  projected  finishing  cost 

APPENDIX  B 

Examples  of  Lists  of  Background  Material  Supplied  to  Peer  Review  Teams 

List  of  Central  Arizona  Project 

7.    Copies  of  report  reviews  published  in  professional 

Background  Documents 

journals,  if  any 

8.    Curation  agreement — Western  Archeological  and 

1.    Section  106,  National  Historic  Preservation  Act 

Conservation  Center  and  Arizona  State  Museum 

case  report 

agreements 

2.    Central  Arizona  Project  Overview  (in  press) 

9.    Copies  of  letters  from  State  Historic  Preservation 

background  and  general  management  plan 

Officers  and  Advisory  Council  on  Historic 

3.    Programmatic  Memorandum  of  Agreement 

Preservation  in  reference  to  their  reviews  of 

4.    Plan  6 

mitigation  activities  in  various  features 

5.    Selected  feature  descriptions  of  archeological 

10.  Maps  of  Central  Arizona  Project 

projects: 

a.    Cave  Creek  feature  (an  early  archeological 

project — Survey  7) 

i.  Case  report 

List  of  Carson  Desert  Background  Documents 

ii.  Memorandum  of  Agreement 

iii.  Scope  of  work 

1 .    Peer  Review  Team 

iv.  Contract 

2.    Preliminary  Agenda 

v.  Product  (report) 

3.    Involved  individuals/organizations 

b.    Granite  Reef  of  Salt  River  Aqueduct  feature 

4.    Plan  of  action  for  cultural  resource  management 

i.  Case  report 

at  Stillwater  Wildlife  Management  Area 

ii.  Memorandum  of  Agreement 

5.    Selected  chapters  from  Preliminary  Investigations 

iii.  Scope  of  work 

in  Stillwater  Marsh:  Human  Prehistory  and 

iv.  Contract 

Geoarcheology,  Volume  1 

v.  Product  (report) 

6.    Osteological  analysis  of  human  remains  removed 

c.    Current  feature  (Waddell  Dam) 

from  Stillwater  Marsh 

i.  Survey  report 

7.    Newlands  Project  information 

ii.  Mitigation  plan 

a.  Secretary  of  the  Interior's  decision  document 

iii.  Memorandum  of  Agreement  or  PMOA 

b.  Operating  criteria  and  piocedures 

iv.  Scope  of  work 

c.  Map  of  Newlands  Project 

v.  Contract 

S     Final  Report  on  1987  Fieldwork  Conducted  on 

vi.  Progress  reports 

Stillwater  Marsh  Sites 

6.    Research  design  for  historic  archeology 
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APPENDIX  C 

Examples  of  "Charges"  to  Peer  Review  Team 


Central  Arizona  Project  Charge 

The  Central  Arizona  Project  archeological  program 
is  currently  the  largest  archeological  rescue  project 
in  the  land.  Its  completion  will  take  several  more 
years  of  field  work,  analysis,  and  report  writing. 
The  object  of  the  review  of  the  program  you  are 
about  to  undertake  is  to  assess  the  project  at  the 
level  of  detail  you  deem  appropriate  in  order  to 
provide  a  written  report  to  the  Departmental 
Consulting  Archeologist  and  the  Bureau  of 
Reclamation.  We  want  to  know  the  weaknesses  as 
well  as  the  strengths  of  the  program.  We  want  you 
to  know  that  we  will  seriously  consider  all  of  the 
recommendations  the  team  may  make  to  improve 
the  program.  Consequently,  no  aspect  of  the 
program  is  off-limits  to  your  review.  You  have 
already  been  provided  with  a  weighty  set  of 
documents  related  to  the  program  and  over  the  next 
couple  of  days  you  will  have  the  opportunity  to 
meet  with  Bureau  of  Reclamation  personnel,  several 
contractors,  representatives  of  the  State  Historic 
Preservation  Officers  and  the  Advisory  Council  on 
Historic  Preservation.  We  have  developed  a  list  of 
topics  in  which  we  think  that  you  will  be 
interested.  Some  of  these  topics  are  covered  in  the 
documents  we  have  provided.  Today  and  tomorrow 
we,  and  others,  will  provide  you  with  additional 
information  as  your  review  progresses. 
Additionally,  we  have  prepared  a  suggested  outline 
for  your  report.  Hopefully  the  efforts  we  have  put 
forth  will  make  your  job  easier  and  your  work 
efficient.  In  closing  we  want  to  emphasize  that  your 
review  of  the  project  is  to  be  thorough  and 
comprehensive.  Be  assured  that  we  are  here  to 
provide  you  with  the  facts  and  the  contexts  in 
which  the  program  has  been  developed  and 
managed. 

Carson  Desert  Archeological  Peer  Review  Charge 

The  objective  of  the  review  of  the  program  you  are 
about  to  undertake  is  to  assess  the  project  at  the 
level  of  detail  you  deem  appropriate  in  order  to 
provide  a  written  report  to  the  Departmental 
Consulting  Archeologist.  We  want  to  know  the 
weaknesses  as  well  as  the  strengths  of  the  program. 
We  want  you  to  know  that  we  will  seriously 


consider  all  of  the  recommendations  the  team  may 
make  to  improve  the  program.  Consequently,  no 
aspect  of  the  program  is  off-limits  to  your  review. 
You  have  already  been  provided  with  a  weighty  set 
of  documents  related  to  the  program  and  over  the 
next  couple  of  days  you  will  have  the  opportunity 
to  meet  with  interested  individuals  and 
organizational  personnel  from  the  Fish  and  Wildlife 
Service,  Bureau  of  Reclamation,  Bureau  of  Land 
Management,  Nevada  State  Museum,  Fallon 
Paiute-Shoshone  Tribe,  Anthropology  Department 
University  of  Reno,  Intermountain  Research, 
Churchill  County  Museum  and  Archives,  Advisory 
Council  on  Historic  Preservation,  and  the  State 
Historic  Preservation  Officer. 

We  have  developed  a  list  of  topics  which  we  suggest 
that  you  review.  Some  of  these  topics  are  covered  in 
the  documents  we  have  provided.  Additionally,  we 
have  prepared  a  suggested  outline  for  the  report. 
Hopefully  the  efforts  we  have  put  forth  will  make 
your  job  easier  and  your  work  efficient. 

I  would  like  to  emphasize  that  your  review  of  the 
project  is  to  be  thorough  and  comprehensive  and  is 
not  restricted  to  the  suggested  topics  if  the  team 
determines  that  other  matters  are  pertinent.  Be 
assured  that  we  are  here  to  provide  you  with  the 
facts  and  the  context  in  which  the  program  has 
been  developed  and  managed. 

We  expect  a  formal  written  report  of  the  Peer 
Review  Team.  This  report  will  constitute  the  Peer 
Review  Team's  official  findings  and 
recommendations.  The  following  outline  is  offered 
only  as  guidance  to  the  Peer  Review  Team.  The 
Peer  Review  Team  may  develop  its  own  report 
format  but  we  expect  that  it  will  provide  objective 
criteria  and  your  evaluation  of  them  by  which  the 
program  can  be  judged  in  respect  to  its  aims,  costs, 
productivity,  quality,  etc. 

Bcnnie  C.  Keel 

Departmental  Consulting  Archeologist 

September  16,  1988 
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APPENDIX  D 

Examples  of  Peer  Review 

Report  Outlines 

Suggested  Outline  for  Peer  Review  Report, 

Proposed  Report  Outline  for 

Central  Arizona  Project 

Carson  Desert  Report 

I.  Introduction 

I. 

Introduction 

II.  General  Evaluation 

n. 

General  Evaluation 

A.  Research  Design (s) 

A.  Research  Design — Appropriateness 

B.  General  Statements,  which  tie  research 

B.  Translation  of  Research  Design  into  Work 

design  and  its  implementation  together 

Elements 

C.  Evaluation  of  Performance 

C.  Evaluation  of  Program 

1.  Bureau  of  Reclamation  Management  and 

1.  Bureau  of  Reclamation  Management  and 

Administration 

Administration 

2.  Contractor  Performance 

2.  Fish  and  Wildlife  Service  Management, 

a.  Field  work 

Administration,  and  Performance 

b.  Laboratory  processing 

a.  Field  work 

c.  Analysis 

b.  Laboratory  processing 

d.  Report  writing  production 

c.  Analysis 

3.  Quality  versus  Cost  versus  Results 

d.  Report  writing  production 

4.  Dissemination  of  Program  Results 

3.  Participation  by  Other  Organizations 

a.  Public 

a.  Field  work 

b.  Scientific  community 

b.  Laboratory  processing 

c.  States 

c.  Analysis 

d.  Other  Federal  land  manager 

d.  Report  writing  production 

5.  Curation  and  Conservation 

4.  Quality  versus  Cost  versus  Results 

6.  Resource  Preservation 

5.  Dissemination  of  Program  Results 

IE.  Findings 

a.  Public 

IV.  Recommendations 

b.  Scientific  community 

c.  States 

d.  Other  Federal  land  managers 

6.  Curation  and  Conservation 

7.  Resource  Preservation 

in. 

Findings,  a  general  summary  of  review 

PV 

Recommendations 
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